# efficient inference

The Phi-4-mini-instruct model undergoes float8 dynamic activation and weight quantization via torchao, achieving 36% VRAM reduction and 15-20% speed improvement on H100 with minimal accuracy impact.

This model is an INT4 quantized version of Mistral-Small-3.1-24B-Instruct-2503, using the GPTQ algorithm to reduce weights from 16-bit to 4-bit, significantly decreasing disk size and GPU memory requirements.

Omost's phi-3-mini model with 128k context length, utilizing fp8 precision.

Omost's released llama-3 model, featuring 8k context length and nf4 quantization.